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Abstract.  Any  formal  method  or  tool  is  almost  certainly  more  often  ap¬ 
plied  in  situations  where  the  outcome  is  failure  (a  counterexample)  rather 
than  success  (a  correctness  proof).  We  present  a  method  for  symbolic 
model  checking  that  can  lead  to  significant  time  and  memory  savings  for 
model-checking  runs  that  fail,  while  occurring  only  a  small  overhead  for 
model-checking  runs  that  succeed.  Our  method  discovers  an  error  as  soon 
as  it  cannot  be  prevented,  which  can  be  long  before  it  actually  occurs;  for 
example,  the  violation  of  an  invariant  may  become  unpreventable  many 
transitions  before  the  invariant  is  violated. 

The  key  observation  is  that  “unpreventability”  is  a  local  property  of 
a  single  module:  an  error  is  unpreventable  in  a  module  state  if  no  en¬ 
vironment  can  prevent  it.  Therefore,  unpreventability  is  inexpensive  to 
compute  for  each  module,  yet  can  save  much  work  in  the  state  explo¬ 
ration  of  the  global,  compound  system.  Based  on  different  degrees  of 
information  available  about  the  environment,  we  define  and  implement 
several  notions  of  “unpreventability,”  including  the  standard  notion  of 
uncontrollability  from  discrete- event  control.  We  present  experimental 
results  for  two  examples,  a  distributed  database  protocol  and  a  wireless 
communication  protocol. 


1  Introduction 

It  has  been  argued  repeatedly  that  the  main  benefit  of  formal  methods  is  falsifi¬ 
cation,  not  verification;  that  formal  analysis  can  only  demonstrate  the  presence 
of  errors,  not  their  absence.  The  fundamental  reason  for  this  is,  of  course,  that 
mathematics  can  be  applied,  inherently,  only  to  an  abstract  formal  model  of 
a  computing  system,  not  to  the  actual  artifact.  Furthermore,  even  when  a  for¬ 
mal  model  is  verified,  the  successful  verification  attempt  is  typically  preceded 
by  many  iterations  of  unsuccessful  verification  attempts  followed  by  model  revi¬ 
sions.  Therefore,  in  practice,  every  formal  method  and  tool  is  much  more  often 
applied  in  situations  where  the  outcome  is  failure  (a  counterexample),  rather 
than  success  (a  correctness  proof). 

Yet  most  optimizations  in  formal  methods  and  tools  are  tuned  towards  suc¬ 
cess.  For  example,  consider  the  use  of  BDDs  and  similar  data  structures  in 
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model  checking.  Because  of  their  canonicity,  BDDs  are  often  most  effective  in 
applications  that  involve  equivalence  checking  between  complex  boolean  func¬ 
tions.  Successful  model  checking  is  such  an  application:  when  the  set  of  reachable 
states  is  computed  by  iterating  image  computations,  successful  termination  is 
detected  by  an  equivalence  check  (between  the  newly  explored  and  the  previously 
explored  states).  By  contrast,  when  model  checking  fails,  a  counterexample  is 
detected  before  the  image  iteration  terminates,  and  other  data  structures,  per¬ 
haps  noncanonical  ones,  may  be  more  efficient  [BCCZ99].  To  point  out  a  second 
example,  much  ink  has  been  spent  discussing  whether  “forward”  or  “backward” 
state  exploration  is  preferable  (see,  e.g.,  [HKQ98]).  If  we  expect  to  find  a  coun¬ 
terexample,  then  the  answer  seems  clear  but  rarely  practiced:  the  simultaneous, 
dove-tailed  iteration  of  images  and  pre-images  is  likely  to  find  the  counterex¬ 
ample  by  looking  at  fewer  states  than  either  unidirectional  method.  Third,  in 
compositional  methods,  the  emphasis  is  almost  invariably  on  how  to  decompose 
correctness  proofs  (see,  e.g.,  [HQR98]),  not  on  how  to  find  counterexamples  by 
looking  at  individual  system  components  instead  of  their  product.  In  this  paper, 
we  address  this  third  issue. 

Consider  a  system  with  two  processes.  The  first  process  waits  on  a  binary 
input  from  the  second  process;  if  the  input  is  0,  it  proceeds  correctly;  if  the 
input  is  1,  it  proceeds  for  n  transitions  before  entering  an  error  state.  Suppose 
the  second  process  may  indeed  output  1.  By  global  state  exploration  (forward 
or  backward) ,  n  - 1-1  iterations  are  necessary  to  encounter  the  error  and  return 
a  counterexample.  This  is  despite  the  fact  that  things  may  go  terribly  wrong, 
without  chance  of  recovery,  already  in  the  first  transition.  We  propose  to  instead 
proceed  in  two  steps.  First,  we  compute  on  each  individual  process  (i.e.,  typically 
on  a  small  state  space)  the  states  that  are  controllable  to  satisfy  the  requirements. 
In  our  example,  the  initial  state  is  controllable  (because  the  environment  may 
output  0  and  avoid  an  error),  but  the  state  following  a  single  1  input  is  not 
(no  environment  can  avoid  the  error).  Second,  on  the  global  state  space,  we 
restrict  search  to  the  controllable  states,  and  report  an  error  as  soon  as  they  are 
left.  In  our  example,  the  error  is  reported  after  a  single  image  (or  pre-image) 
computation  on  the  global  state  space.  (A  counterexample  can  be  produced 
from  this  and  the  precomputed  controllability  information  of  the  first  process.) 
Note  that  both  steps  are  fully  automatic.  Moreover,  the  lower  number  of  global 
iterations  usually  translates  into  lower  memory  requirements,  because  BDD  size 
often  grows  with  the  number  of  iterations.  Finally,  when  no  counterexample  is 
found,  the  overhead  of  our  method  is  mostly  in  performing  step  1,  which  does 
not  involve  the  global  state  space  and  therefore  is  usually  uncritical. 

We  present  several  refinements  of  this  basic  idea,  and  demonstrate  the  effi¬ 
ciency  of  the  method  with  two  examples,  a  distributed  database  protocol  and  a 
wireless  communication  protocol.  In  the  first  example,  there  are  two  sites  that 
can  sell  and  buy  back  seats  on  the  same  airplane  [BGM92].  The  protocol  aims  at 
ensuring  that  no  more  seats  are  sold  than  the  total  available,  while  enabling  the 
two  sites  to  exchange  unsold  seats,  in  case  one  site  wishes  to  sell  more  seats  than 
initially  allotted.  The  second  example  is  from  the  Two-Chip  Intercom  (TCI) 
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project  of  the  Berkeley  Wireless  Research  Center  [BWR].  The  TCI  network  is 
a  wireless  local  network  which  allows  approximately  40  remotes,  one  for  each 
user,  to  transmit  voice  with  point-to-point  and  broadcast  communication.  The 
operation  of  the  network  is  coordinated  by  a  base  station,  which  assigns  chan¬ 
nels  to  the  users  through  a  TDMA  scheme.  In  both  examples,  we  first  found 
errors  that  occurred  in  our  initial  formulation  of  the  models,  and  then  seeded 
bugs  at  random.  Our  methods  succeeded  in  reducing  the  number  of  global  image 
computation  steps  required  for  finding  the  errors,  often  reducing  the  maximum 
number  of  BDD  nodes  used  in  the  verification  process.  The  methods  are  partic¬ 
ularly  effective  when  the  BDDs  representing  the  controllable  states  are  small  in 
comparison  to  the  BDD  representing  the  set  of  reachable  states. 

To  explain  several  fine  points  about  our  method,  we  need  to  be  more  formal. 
To  study  the  controllability  of  a  module  P,  we  consider  a  game  between  P 
and  its  environment:  the  moves  of  P  consist  in  choosing  new  values  for  the 
variables  controlled  by  P;  the  moves  of  the  environment  of  P  consist  in  choosing 
new  values  for  the  input  variables  of  P.  A  state  s  of  P  is  controllable  with 
respect  to  the  invariant  Up  if  the  environment  has  a  strategy  that  ensures  that 
p  always  holds.  Hence,  if  a  state  s  is  not  controllable,  we  know  that  P  from 
s  can  reach  a  -instate,  regardless  of  how  the  environment  behaves.  The  set 
Cp  of  controllable  states  of  P  can  be  computed  iteratively,  using  the  standard 
algorithm  for  solving  safety  games,  which  differs  from  backward  reachability  only 
in  the  definition  of  the  pre-image  operator.  Symmetrically,  we  can  compute  the 
set  Cq  of  controllable  states  of  Q  w.r.t.  Up.  Then,  instead  of  checking  that 
P  ||  Q  stays  within  the  invariant  Up,  we  check  whether  P  ||  Q  stays  within  the 
stronger  invariant  □(Cp  A  Cq).  As  soon  as  P  ||  Q  reaches  a  state  s  that  violates 
a  controllability  predicate,  say,  Cp,  by  retracing  the  computation  of  Cp,  taking 
into  account  also  Q,  we  can  construct  a  path  of  P  ||  Q  from  s  to  a  state  t  that 
violates  the  specification  p.  Together  with  a  path  from  an  initial  state  to  s,  this 
provides  a  counterexample  to  Up.  While  the  error  occurs  only  at  t,  we  detect  it 
already  at  s,  as  soon  as  it  cannot  be  prevented.  The  method  can  be  extended  to 
arbitrary  LTL  requirements. 

The  notion  of  controllability  defined  above  is  classical,  but  it  is  often  not 
strong  enough  to  enable  the  early  detection  of  errors.  To  understand  why,  con¬ 
sider  an  invariant  that  relates  a  variable  x  in  module  P  with  a  variable  y  in 
module  Q,  for  example  by  requiring  that  x  =  y,  and  assume  that  y  is  an  input 
variable  to  P.  Consider  a  state  s,  in  which  module  P  is  about  to  change  the 
value  of  x  without  synchronizing  this  change  with  Q.  Intuitively,  it  seems  obvi¬ 
ous  that  such  a  change  can  break  the  invariant,  and  that  the  state  should  not  be 
considered  controllable  (how  can  Q  possibly  know  that  this  is  going  to  happen, 
and  change  the  value  of  y  correspondingly?).  However,  according  to  the  classical 
definition  of  controllability,  the  state  s  is  controllable:  in  fact,  the  environment 
has  a  move  (changing  the  value  of  y  correspondingly)  to  control  P.  This  ex¬ 
ample  indicates  that  in  order  to  obtain  stronger  (and  more  effective)  notions  of 
controllability,  we  need  to  compute  the  set  of  controllable  states  by  taking  into 
account  the  real  capabilities  of  the  other  modules  composing  the  system.  We 
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introduce  three  such  stronger  notions  of  controllability:  constrained,  lazy,  and 
bounded  controllability.  Our  experimental  results  demonstrate  that  there  is  a 
distinct  advantage  in  using  these  stronger  notions  of  controllability. 

Lazy  controllability  can  be  applied  to  systems  in  which  all  the  modules  are 
lazy,  i.e.,  if  the  modules  always  have  the  option  of  leaving  unchanged  the  val¬ 
ues  of  the  variables  they  control  [AH99].  Thus,  laziness  models  the  assumption 
of  speed  independence,  and  is  used  heavily  in  the  modeling  of  asynchronous 
systems.  If  the  environment  is  lazy,  then  there  is  no  way  of  preventing  the  envi¬ 
ronment  from  always  choosing  its  “stutter”  move.  Hence,  we  can  strengthen  the 
definition  of  controllability  by  requiring  that  the  stutter  strategy  of  the  environ¬ 
ment,  rather  than  an  arbitrary  strategy,  must  be  able  to  control.  In  the  above 
example,  the  state  s  of  module  P  is  clearly  not  lazily  controllable,  since  a  change 
of  x  cannot  be  controlled  by  leaving  y  unchanged.  Constrained  controllability  is 
a  notion  of  controllability  that  can  be  used  also  when  the  system  is  not  lazy. 
Constrained  controllability  takes  into  account,  in  computing  the  sets  of  control¬ 
lable  states,  which  moves  are  possible  for  the  environment.  To  compute  the  set 
of  constrainedly  controllable  states  of  a  module  P,  we  construct  a  transition  rela¬ 
tion  that  constrains  the  moves  of  the  environment.  This  is  done  by  automatically 
abstracting  away  from  the  transition  relations  of  the  other  modules  the  variables 
that  are  not  shared  by  P.  We  then  define  the  controllable  states  by  considering 
a  game  between  P  and  a  so  constrained  environment.  Finally,  bounded  controlla¬ 
bility  is  a  notion  that  can  again  be  applied  to  any  system,  and  it  generalizes  both 
lazy  and  constrained  controllability.  It  considers  environments  that  have  both  a 
set  of  unavoidable  moves  (such  as  the  lazy  move  for  lazy  systems),  and  possible 
moves  (by  considering  constraints  to  the  moves,  similarly  to  constrained  con¬ 
trollability).  We  also  introduce  a  technique  called  iterative  strengthening,  which 
can  be  used  to  strengthen  any  of  these  notions  of  controllability.  In  essence,  it 
is  based  on  the  idea  that  a  module,  in  order  to  control  another  module,  cannot 
use  a  move  that  would  cause  it  to  leave  its  own  set  of  controllable  states. 

It  is  worth  noting  that  the  technqiues  developed  in  this  paper  can  also  be  used 
in  an  informal  verification  environment:  after  computing  the  uncontrollability 
states  for  each  of  the  components,  one  can  simulate  the  design  and  check  if  any 
of  these  uncontrollable  states  can  be  reached.  This  is  similar  to  the  techniques 
retrograde  analysis  [JSAA97],  or  target  enlargement  [YD98]  in  simulation.  The 
main  idea  of  retrograde  analysis  and  target  enlargement  is  that  the  set  of  states 
that  violate  the  invariants  are  “enlarged”  with  their  preimages,  and  hence  the 
chances  of  hitting  this  enlarged  set  is  increased.  Our  techniques  not  only  add 
modularity  in  the  computation  of  target  enlargemen,  they  also  allow  one  to 
detect  the  violation  of  liveness  properties  through  simulation. 

The  algorithmic  control  of  reactive  systems  has  been  studied  extensively 
before  (see,  e.g.,  [RW89,EJ91,Tho95]).  However,  the  use  of  controllability  in 
automatic  verification  is  relatively  new  (see,  e.g.,  [KV96,AHK97,AdAHM99]). 
The  work  closest  to  ours  is  [ASSSV94],  where  transition  systems  for  components 
are  minimized  by  taking  into  account  if  a  state  satisfies  or  violates  a  given  CTL 
property  under  all  environments.  In  [Dil88],  autofailure  captures  the  concept 
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that  no  environment  can  prevent  failure  and  is  used  to  compare  the  equivalence 
of  asynchronous  circuits. 

2  Preliminaries 

Given  a  set  V  of  typed  variables,  a  state  s  over  V  is  an  assignment  for  V  that 
assigns  to  each  x  £  V  a  value  s[x].  We  indicate  with  States(V)  be  the  set  of 
all  states  over  V,  and  with  V(V)  the  set  of  predicates  over  V.  Furthermore,  we 
denote  by  V'  =  {x1  \  x  £  V}  the  set  obtained  by  priming  each  variable  in  V. 
Given  a  predicate  H  £  V(V),  we  denote  by  H'  €  V(V')  the  predicate  obtained 
by  replacing  in  H  every  x  €  V  with  x'  £  V'.  A  module  P  =  ( Cp,£p,Ip,Tp ) 
consists  of  the  following  components: 

1.  A  (finite)  set  Cp  of  controlled  variables,  each  with  finite  domain,  consisting 
of  the  variables  whose  values  can  be  accessed  and  modified  by  P. 

2.  A  (finite)  set  £p  of  external  variables,  each  with  finite  domain,  consisting  of 
the  variables  whose  values  can  be  accessed,  but  not  modified,  by  P. 

3.  A  transition  predicate  TP  £  V(Cp  U  £p  U  C'P). 

4.  An  initial  predicate  Ip  £  V{Cp). 

We  denote  by  Vp  =  Cp  Li  £p  the  set  of  variables  mentioned  by  the  module. 
Given  a  state  s  over  Vp,  we  write  s  |=  Ip  if  Ip  is  satisfied  under  the  variable 
interpretation  specified  by  s.  Given  two  states  s,  s'  over  Vp,  we  write  («,*')  \=Tp 
if  predicate  TP  is  satisfied  by  the  interpretation  that  assigns  to  x  £  Vp  the 
value  s[x],  and  to  x'  £  V'P  the  value  s'[x].  A  module  P  is  non-blocking  if  the 
predicate  Ip  is  satisfiable,  i.e.,  if  the  module  has  at  least  one  initial  state,  and  if 
the  assertion  VVp  .  3 CP  .  Tp  holds,  so  that  every  state  has  a  successor. 

A  trace  of  module  P  is  a  finite  sequence  of  states  so,  «i,  «2,  ■  ■  ■  sn  £ 
States(Vp),  where  n  >  0  and  (s*,,  s*,+i)  |=  Tp  for  all  0  <  k  <  n;  the  trace 
is  initial  if  so  (=  Ip.  We  denote  by  jC{P)  the  set  of  initial  traces  of  module  P. 
For  a  module  P,  we  consider  specifications  expressed  by  linear-time  temporal 
logic  (LTL)  formulas  whose  atomic  predicates  are  in  V(Vp).  As  usual,  given  an 
LTL  formula  tp,  we  write  P  f=  tp  iff  a  |=  tp  for  all  a  £  £(P). 

Two  modules  P  and  Q  are  composable  if  Cp  FI  Cq  =  0;  in  this  case,  their 
parallel  composition  P\\Q  is  defined  as:  P  ||  Q  =  (Cp  U  Cq,  ( £p  U  £q)  \  {Cp  U 
Cq),  Ip  A  Iq,Tp  A  Tq)  .  Note  that  composition  preserves  non-blockingness. 

We  assume  that  all  predicates  are  represented  in  such  a  way  that  boolean 
operations  and  existential  quantification  of  variables  are  efficiently  computable. 
Likewise,  we  assume  that  satisfiability  of  all  predicates  can  be  checked  efficiently. 
Binary  decision  diagrams  (BDDs)  provide  a  suitable  representation  [Bry86]. 

Controllability.  We  can  view  the  interaction  between  a  module  P  and  its  envi¬ 
ronment  as  a  game.  At  each  round  of  the  game,  the  module  P  chooses  the  next 
values  for  controlled  variables  Cp,  while  the  environment  chooses  the  next  values 
for  the  external  variables  £p.  Given  an  LTL  specification  ip,  we  say  that  a  state 
s  of  P  is  controllable  with  respect  to  tp  if  the  environment  can  ensure  that  all 
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traces  from  s  satisfy  p.  To  formalize  this  definition,  we  use  the  notion  of  strat¬ 
egy.  A  module  strategy  n  for  P  is  a  mapping  n  :  States(Vp)+  States(Cp ) 
that  maps  each  finite  sequence  so,  si, . . . ,  Sk  of  module  states  into  a  state 
7r(so,  Si,  . . . ,  Sfc)  such  that  (sk,  n(so,  Si,  ■  ■  ■ ,  Sfc))  |=  Tp.  Similarly,  an  environ¬ 
ment  strategy  77  for  P  is  a  mapping  r)  :  States(Vp)+  States(£p)  that  maps 
each  finite  sequence  of  module  states  into  a  state  specifying  the  next  values 
of  the  external  variables.  Given  two  states  si  and  s 2  over  two  disjoint  sets 
of  variables  Vi  and  V2,  we  denote  by  si  X  s 2  the  state  over  Vi  U  V2  that 
agrees  with  si  and  s 2  over  the  common  variables.  With  this  notation,  for  all 
s  £  StatesiVp)  and  all  module  strategies  n  and  environment  strategies  77,  we 
define  Outcome(s , n , rj)  £  States(Vp)u  to  be  the  trace  so>si,S2j---  defined  by 
so  =  s  and  by  s^+i  =  n(so,  «i,  ■  ■  ■ ,  «*)  X  77(50,  «i, . . . ,  Sk )•  Given  an  LTL  formula 
p  over  Vp,  we  say  that  a  state  s  £  States (Vp)  is  controllable  with  respect  to  p 
iff  there  is  an  environment  strategy  77  such  that,  for  every  module  strategy  n,  we 
have  Outcome(s,Tr,T])  |=  p.  We  let  Ctr{P,p)  be  the  predicate  over  Vp  defining 
the  set  of  states  of  P  controllable  with  respect  to  p. 

Roughly,  a  state  of  P  is  controllable  w.r.t.  p  exactly  when  there  is  an  envi¬ 
ronment  E  for  P  such  that  all  paths  from  s  in  P  ||  E  satisfy  p.  Since  in  general 
E  can  contain  variables  not  in  P,  to  make  the  above  statement  precise  we  need 
to  introduce  the  notion  of  extension  of  a  state.  Given  a  state  s  over  V  and  a 
state  t  over  U ,  with  V  C  U,  we  say  that  t  is  an  extension  of  s  if  s[x]  =  t\x\  for 
all  x  £  V.  Then,  there  is  module  E  composable  with  P  such  that  all  paths  from 
extensions  of  s  in  P  ||  E  satisfy  p  iff  s  £  Ctr(P ,  p)  [AdAHM99j. 

3  Early  Detection  of  Invariant  Violation 

Forward  and  backward  state  exploration.  Given  a  module  R  and  a  predi¬ 
cate  p  over  Vp,  the  problem  of  invariant  verification  consists  in  checking  whether 
R  |=  Up.  We  can  solve  this  problem  using  classic  forward  or  backward  state  ex¬ 
ploration.  Forward  exploration  starts  with  the  set  of  initial  states  of  R,  and 
iterates  a  post-image  computation,  terminating  when  a  state  satisfying  -up  has 
been  reached,  or  when  the  set  of  reachable  states  of  R  has  been  computed.  In  the 
first  case  we  conclude  R  Up-,  in  the  second,  R  |=  Up.  Backward  exploration 
starts  with  the  set  -1  p  of  states  violating  the  invariant,  and  iterates  a  pre-image 
computation,  terminating  when  a  state  satisfying  Ip  has  been  reached,  or  when 
the  set  of  all  states  that  can  reach  -<p  has  been  computed.  Again,  in  the  first  case 
we  conclude  R  ^  Up  and  in  the  second  R  |=  Up.  If  the  answer  to  the  invariant 
verification  question  is  negative,  these  algorithms  can  also  construct  a  counterex¬ 
ample  So,  ■■■  ,sm  of  minimal  length  leading  from  s0  |=  Ir  to  sm  (=  ~<p,  and  such 
that  for  0  <  f  <  777  we  have  (s* ,  gj+i)  |=  Tp.  If  our  aim  is  to  find  counterexamples 
quickly,  an  algorithm  that  alternates  forward  and  backward  reachability  is  likely 
to  explore  fewer  states  than  the  two  unidirectional  algorithms.  The  algorithm 
alternates  post-image  computations  starting  from  Ip  with  pre-image  computa¬ 
tions  starting  from  -1  p,  terminating  as  soon  as  the  post  and  pre- images  intersect, 
or  as  soon  as  a  fixpoint  is  reached.  We  denote  any  of  these  three  algorithms  (or 
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variations  thereof)  by  InvCheck(R,p).  We  assume  that  InvCheck{R,p )  returns 
answer  Yes  or  No,  depending  on  whether  R  |=  Up  or  R  Up,  along  with  a 
counterexample  in  the  latter  case. 

Controllability  and  early  error  detection.  Given  n  >  1  modules 

Pi,P2,. .  .,Pn  and  a  predicate  < p  €  PdJiLi  VpJ,  the  modular  version  of  the  invari¬ 
ant  verification  problem  consists  in  checking  whether  Pi  ||  •••  \\Pn  |=  Up.  We 
can  use  the  notion  of  controllability  to  try  to  detect  a  violation  of  the  invariant 
p  in  fewer  iterations  of  post  or  pre-image  computation  than  the  forward  and 
backward  exploration  algorithms  described  above.  The  idea  is  to  pre-compute 
the  states  of  each  module  P\ , . . .. ,  Pn  that  are  controllable  w.r.t.  Up.  We  can 
then  detect  a  violation  of  the  invariant  as  soon  as  we  reach  a  state  s  that  is  not 
controllable  for  some  of  the  modules,  rather  than  waiting  until  we  reach  a  state 
actually  satisfying  -up.  In  fact,  we  know  that  from  s  there  is  a  path  leading  to 
-up  in  the  global  system:  for  this  reason,  if  a  state  is  not  controllable  for  some  of 
the  modules,  we  say  that  the  state  is  doomed. 

To  implement  this  idea,  let  R  =  Pi  ||  ■  ■  ■  ||  P„,  and  for  1  <  i  <  n,  let 
absi(p)  =  3(Vp  \  VPi)  .  p  be  an  approximation  of  p  that  involves  only  the 
variables  of  Pp  note  that  p  -»  absi(p).  For  each  1  <  i  <  n,  we  can  compute 
the  set  Ctr(Pi,Dabsi(p))  of  controllable  states  of  Pi  w.r.t.  Uabsi(p)  using  a 
classical  algorithm  for  safety  games.  For  a  module  P,  the  algorithm  uses  the 
uncontrollable  predecessor  operator  UPrep  :  V(Vp)  i-»  V(Vp),  defined  by 

UPr eP(X)  =  VS'p  .  3C'P  .  (: TP  A  X')  . 

The  predicate  UPrep  (X)  defines  the  set  of  states  from  which,  regardless  of  the 
move  of  the  environment,  the  module  P  can  resolve  its  internal  nondeterminism 
to  make  X  true.  Note  that  a  quantifier  switch  is  required  to  to  compute  the 
uncontrollable  predecessors,  as  opposed  to  the  computations  of  pre- images  and 
post-images,  where  where  only  existential  quantification  is  required.  For  a  mod¬ 
ule  P  and  an  invariant  Up,  we  can  compute  the  set  Ctr(P,  Up)  of  controllable 
states  of  P  with  respect  to  Up  by  letting  f/0  =  p ,  and  for  k  >  0,  by  letting 

Uk  =  ^P  V  UPrep(Pfc_i),  (1) 

until  we  have  Uk  =  Uk- 1,  at  which  point  we  have  Ctr(P,  Up)  =  -,t4.  For  k  >  0 
the  set  Uk  consists  of  the  states  from  which  the  environment  cannot  prevent 
module  P  from  reaching  -<p  in  at  most  k  steps.  Note  that  for  all  1  <  i  <  n,  the 
computation  of  Ctr(Pj,Uabsi(p))  is  carried  out  on  the  state  space  of  module 
Pi,  rather  than  on  the  (larger)  state  space  of  the  complete  system.  We  can  then 
solve  the  invariant  checking  problem  Pi  ||  •  •  •  ||  Pn  |=  Up  by  executing 

n 

InvCheck[P\  ||  •  •  •  ||  Pn,  Pt\  f\  Ctr{Pi,Uabsi(p)))  .  (2) 

i= 0 

It  is  necessary  to  conjoin  p  to  the  set  of  controllable  states  in  the  above  check, 
because  for  1  <  i  <  n,  predicate  absi(p)  (and  thus,  possibly,  Ctr(Pi ,  Uabsi(p))) 
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may  be  weaker  than  p.  If  check  (2)  returns  answer  Yes,  then  we  have  immedi¬ 
ately  that  Pi  ||  •  •  •  ||  Pn  |=  Up.  If  the  check  returns  answer  No,  we  can  conclude 
that  Pi  ||  •  •  •  ||  Pn  □¥>■  In  this  latter  case,  the  check  (2)  also  returns  a  partial 
counterexample  so,  si, . . . ,  sm,  with  sm  ft  Ctr(Pj,  Upj)  for  some  1  <  j  <  n. 
If  sm  |=  this  counterexample  is  also  a  counterexample  to  Up.  Otherwise, 
to  obtain  a  counterexample  so,  ■  ■  ■  j  sm,sm+ 1, . . . ,  sm+r  with  sm+r  p,  we  pro¬ 
ceed  as  follows.  Let  Uq.  U\. ...,  Uk  be  the  predicates  computed  by  Algorithm  1 
during  the  computation  of  Ctr(Pj,  Upj)\  note  that  sm  |=  Uk ■  For  l  >  0,  given 
Sm  t  /  i ,  we  pick  Sr77,-|-/  such  that  sm+i  | —  U^—i  and  (smpi—i,smpi)  /\ii  O', ■ 
The  process  terminates  as  soon  as  we  reach  an  l  such  that  sm+i  |=  -<p:  since  the 
implication  P0  — »  ~^P  holds,  this  will  occur  in  at  most  k  steps. 

4  Lazy  and  Constrained  Controllability 

In  the  previous  section,  we  have  used  the  notion  of  controllability  to  compute 
sets  of  doomed  states,  from  which  we  know  that  there  is  a  path  violating  the 
invariant.  In  order  to  detect  errors  early,  we  should  compute  the  largest  possible 
sets  of  doomed  states.  To  this  end,  we  introduce  two  notions  of  controllability 
that  can  be  stronger  than  the  classical  definition  of  the  previous  section.  The 
first  notion,  lazy  controllability,  can  be  applied  to  systems  that  are  composed 
only  of  lazy  modules,  i.e.  of  modules  that  need  not  react  to  their  inputs.  Several 
communication  protocols  can  be  modeled  as  the  composition  of  lazy  modules. 
The  second  notion,  constrained  controllability,  can  be  applied  to  any  system. 

Lazy  controllability.  A  module  is  lazy  if  it  always  has  the  option  of  leav¬ 
ing  its  controlled  variables  unchanged.  Formally,  a  module  P  is  lazy  if  we  have 
( s ,  s)  |=  Tp  for  every  state  s  over  Vp.  If  all  the  modules  composing  the  system  are 
lazy,  then  we  can  re-examine  the  notion  of  controllability  described  in  Section  3 
to  take  into  account  this  fact.  Precisely,  we  defined  a  state  to  be  controllable 
w.r.t.  an  LTL  property  p  if  there  is  a  strategy  for  the  environment  to  ensure 
that  the  resulting  trace  satisfies  p,  regardless  of  the  strategy  used  by  the  system. 
But  if  the  environment  is  lazy,  we  must  always  account  for  the  possibility  that 
the  environment  plays  according  to  its  lazy  strategy,  in  which  the  values  of  the 
external  variables  of  the  module  never  change.  Hence,  if  all  modules  are  lazy, 
there  is  a  second  condition  that  has  to  be  satisfied  for  a  state  to  be  controllable: 
for  every  strategy  of  the  module,  the  lazy  environment  strategy  should  lead  to 
a  trace  that  satisfies  p.  It  is  easy  to  see,  however,  that  this  second  condition 
for  controllability  subsumes  the  first.  We  can  summarize  these  considerations 
with  the  following  definition.  For  1  <  i  <  n,  denote  by  rf  the  lazy  environ¬ 
ment  strategy  of  module  Pi,  which  leaves  the  values  of  the  external  variables 
of  Pi  always  unchanged.  We  say  that  a  state  s  €  States  (Vp;)  is  lazily  control¬ 
lable  with  respect  to  a  LTL  formula  ip  iff,  for  every  module  strategy  n,  we  have 
Outcome(s,  n,  rf)  |=  p.  We  let  LCtr(P,  p)  be  the  predicate  over  Vp  defining  the 
set  of  states  of  P  that  are  lazily  controllable  with  respect  to  p. 

We  can  compute  for  the  invariant  Up  the  predicate  LCtr(P,  Up)  by  replacing 
the  operator  UPre  in  Algorithm  1  with  the  operator  LUPre  :  V(Vp)  i-»  V(Vp), 
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the  lazily  uncontrollable  predecessor  operator ,  defined  by: 

LUPreP(A)  =  3 C'P  .  (TP  A  X')[£P/£'P\  . 

where  ( TP  A  X')[£P/£'P\  is  obtained  from  TP  A  X'  by  replacing  each  variable 
x1  £  £'P  with  x  €  £p.  Note  that  LUPrepA  computes  a  superset  of  UPrepA,  and 
therefore  the  set  LCtr(P ,  Uip)  of  lazily  controllable  states  is  always  a  subset  of 
the  controllable  states  Ctr(P,  □</?)• 

Given  n  >  1  lazy  modules  Pi,P2, . . .  ,Pn  and  a  predicate  ip  €  'POJJL.,  VpJ,  let 
R  =  Pi  ||  •  •  •  ||  Pn,  and  for  all  1  <  i  <  n.  We  can  check  whether  Pi  ||  •  •  •  ||  Pn  |= 
Up>  by  executing  InvCheck(R,  ip/\  /\"=1  LCtr(Pi,  □  absi(ip))).  If  this  check  returns 
answer  No,  we  can  construct  a  counterexample  to  Uip  as  in  Section  3. 

Constrained  controllability.  Consider  again  n  >  1  modules  P\  ,  P>, . . . ,  Pn . 
together  with  a  predicate  ip  €  P(UjLi  VpJ.  In  Section  3,  we  defined  a  state  to 
be  controllable  if  it  can  be  controlled  by  an  unconstrained  environment,  which 
can  update  the  external  variables  of  the  module  in  an  arbitrary  way.  However, 
in  the  system  under  consideration,  the  environment  of  a  module  Pi  is  Qi  = 
Pi  ||  •  •  •  II  Pi-i  II  Pi+i  II  •  •  •  II  Pm  for  1  <  i  <  n.  This  environment  cannot  update 
the  external  variables  of  Pi  in  an  arbitrary  way,  but  is  constrained  in  doing  so  by 
the  transition  predicates  of  modules  Pj,  for  1  <  j  <  n,  j  i.  If  we  compute  the 
controllability  predicate  with  respect  to  the  most  general  environment,  instead 
of  Q i,  we  are  giving  to  the  environment  in  charge  of  controlling  Pi  more  freedom 
than  it  really  has.  To  model  this  restriction,  we  can  consider  games  in  which  the 
environment  of  Pi  is  constrained  by  a  transition  predicate  over  VPi  U£'P.  that  over¬ 
approximates  the  transition  predicate  of  Qi.  We  rely  on  an  over-approximation 
to  avoid  mentioning  all  the  variables  in  |J"=1  ,  since  this  would  enlarge  the 

state  space  on  which  the  controllability  predicate  is  computed. 

These  considerations  motivate  the  following  definitions.  Consider  a  module  P 
together  with  a  transition  predicate  H  over  VP  U  £P.  An  H -constrained  strategy 
for  the  environment  of  P  is  a  strategy  r]  :  States (Vp)+  h*  States(£P)  such 
that,  for  all  «o>  *i>.*  •  s*  €  States(VP)+,  we  have  (.s*, '//(.so-  *i ,  -  •  • ,  **))  |=  H- 
Given  an  LTL  formula  (/?  over  VP,  we  say  that  a  state  s  £  States  (Vp)  is  H- 
controllable  if  there  is  an  P-constrained  environment  strategy  r]  such  that,  for 
every  module  strategy  7r,  we  have  Outcome(s,Tr,r |=  ip.  We  let  CCtr(P,  {{Pt))tp) 
be  the  predicate  over  VP  defining  the  set  of  if -controllable  states  of  P  w.r.t. 
ip.1  For  invariant  properties,  the  predicate  CCtr(P,  ({ H))dp )  can  be  computed 
by  replacing  in  Algorithm  1  the  operator  UPre  with  the  operator  CUPrep[P]  : 
V(Vp)  P(Vp),  defined  by: 

CUPreP[P](X)  =  V£'P  .  ( H  3 C'P  .  {TP  A  A"))  . 

When  H  =  true,  CUPrep[P](A)  =  UPrep(Ar);  for  all  other  stronger  predicates 
H,  the  P-uncontrollable  predecessor  operator  CUPrep[P](A)  will  be  a  superset 

1  If  Eh  is  a  module  composable  with  P  having  transition  relation  H,  the  predicate 
CCtr(P,  {(H)) ip)  defines  exactly  the  same  set  of  states  as  the  ATL  formula  {{E))Uip 
interpreted  over  P||  Eh  [AHK97]. 
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of  UPrep(X),  and  therefore  the  set  CCtr(P,  {{ H))(p )  of  U-controllable  states  will 
be  a  subset  of  the  controllable  states  Ctr(P,  □<£). 

Given  a  system  R  =  P1WP2W  . . .  \\Pn  and  a  predicate  €  V(Vr),  for  1  <  i  <  n 
we  let 

where  Uj  =  Vp,  \  Vp;.  We  can  then  check  whether  R  |=  Upi  by  executing 
InvCheck(R,  CCtr(Pi ,  {{ Hj))Uabsi(p> ))).  If  this  check  returns  answer  No, 

we  can  construct  a  counterexample  proceeding  as  in  Section  3. 

5  Experimental  Results 

We  applied  our  methods  for  early  error  detection  to  two  examples:  a  distributed 
database  protocol  and  a  wireless  communication  protocol.  We  implemented  all 
algorithms  on  top  of  the  model  checker  Mocha  [AHM+98],  which  relies  on  the 
BDD  package  and  image  computation  engine  provided  by  VIS  [BIISV+96], 

Demarcation  protocol.  The  demarcation  protocol  is  a  distributed  protocol 
for  maintaining  numerical  constraints  between  distributed  copies  of  a  database 
[BGM92].  We  considered  an  instance  of  the  protocol  that  manages  two  sites  that 
sell  and  buy  back  seats  on  the  same  airplane;  each  site  is  modeled  by  a  module. 
In  order  to  minimize  communication,  each  site  maintains  a  demarcation  variable 
indicating  the  maximum  number  of  seats  it  can  sell  autonomously;  if  the  site 
wishes  to  sell  more  seats  than  this  limit,  it  enters  a  negotiation  phase  with  the 
other  site.  The  invariant  states  that  the  total  number  of  seats  sold  is  always  less 
than  the  total  available. 

In  order  to  estimate  the  sensitivity  of  our  methods  to  differences  in  modeling 
style,  we  wrote  three  models  of  the  demarcation  protocol;  the  models  differ  in 
minor  details,  such  as  the  maximum  number  of  seats  that  can  be  sold  or  bought 
in  a  single  transaction,  or  the  implementation  of  the  communication  channels. 
In  all  models,  each  of  the  two  modules  controls  over  20  variables,  and  has  8-10 
external  variables;  the  diameter  of  the  set  of  reachable  states  is  between  80 
and  120.  We  present  the  number  of  iterations  required  for  finding  errors  in  the 
three  models  using  the  various  notions  of  controllability  in  Table  1.  Some  of  the 
errors  occurred  in  the  formulation  of  the  models,  others  were  seeded  at  random. 

Two-chip  intercom.  The  second  example  is  from  the  Two-Chip  Intercom 
(TCI)  project  of  the  Berkeley  Wireless  Research  Center  [BWR].  TCI  is  a  wire¬ 
less  local  network  which  allows  approximately  40  remotes  to  transmit  voice  with 
point-to-point  and  broadcast  communication.  The  operation  of  the  network  is 
coordinated  by  a  base  station,  which  assigns  channels  to  the  remotes  through  a 
TDMA  scheme.  Each  remote  and  base  station  will  be  implemented  in  a  two-chip 
solution,  one  for  the  digital  component  and  one  for  the  analog.  The  TCI  protocol 
involves  four  layers:  the  functional  layer  (UI),  the  transport  layer,  the  medium 
access  control  (MAC)  layer  and  the  physical  layer.  The  UI  provides  an  interface 
between  the  user  and  the  remote.  The  transport  layer  accepts  service  requests 
from  the  UI,  defines  the  corresponding  messages  to  be  transmitted  across  the 
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(a)  Model  1.  (b)  Model  2.  (c)  Model  3. 

Table  1.  Number  of  iterations  required  in  global  state  exploration  to  find  errors  in 
3  models  of  the  demarcation  protocol.  The  errors  are  el,...,e4.  The  columns  are  L 
(lazy  controllability),  C  (constrained  controllability),  R  (regular  controllability),  and 
G  (traditional  global  state  exploration). 


network,  and  transmits  the  messages  in  packets.  The  transport  layer  also  accepts 
and  interprets  the  incoming  packets  and  sends  the  messages  to  the  UI.  The  MAC 
layer  implements  the  TDMA  scheme.  The  protocol  stack  for  a  remote  is  shown 
in  Figure  1(a).  Each  of  these  blocks  are  described  by  the  designers  in  Esterel 
and  modeled  in  Polis  using  Codesign  Finite  State  Machines  [BCG+97]. 

There  are  four  main  services  available  to  a  user:  ConnReq,  AddReq,  RemReq 
and  DiscReq.  To  enter  the  network,  a  remote  sends  a  connection  request,  Con¬ 
nReq,  together  with  the  id  of  the  remote,  to  the  base  station.  The  base  station 
checks  that  the  remote  is  not  already  registered,  and  that  there  is  a  free  time-slot 
for  the  remote.  It  then  registers  the  remote,  and  sends  a  connection  grant  back 
to  the  the  remote.  If  a  remote  wishes  to  leave  the  network,  it  sends  DiscReq  to 
the  base  station,  which  unregisters  the  remote.  If  two  or  more  remotes  want  to 
start  a  conference,  one  of  them  sends  AddReq  to  the  base  station,  together  with 
the  id’s  of  the  remotes  with  which  it  wants  to  communicate.  The  base  station 
checks  that  the  remotes  are  all  registered,  and  sends  to  each  of  these  remotes  an 
acknowledgment  and  a  time-slot  assignment  for  the  conference.  When  a  remote 
wishes  to  leave  the  conference,  it  sends  a  RemReq  request  to  the  base  station, 
which  reclaims  the  time  slot  allocated  to  the  remote. 

We  consider  a  TCI  network  involving  one  remote  and  one  base  station.  The 
invariant  states  that  if  a  remote  believes  that  it  is  connected  to  the  network, 
then  the  base  station  has  this  remote  registered.  This  property  involves  the 
functional  and  transport  layers.  In  our  experiment,  we  model  the  network  in 
reactive  modules  [AH99]  The  modules  that  model  the  functional  and  transport 
layers  for  both  the  remote  and  the  base  station  are  translated  directly  from  the 
corresponding  CFSM  models;  based  on  the  protocol  specification,  we  provide 
abstractions  for  the  MAC  layer  and  physical  layer  as  well  as  the  channel  between 
the  remote  and  the  base  station.  Due  to  the  semantics  of  CFSM,  the  modules  are 
lazy,  and  therefore,  lazy  controllability  applies.  The  final  model  has  83  variables. 
The  number  of  iterations  required  to  discover  the  various  errors,  some  incurred 
during  the  modeling  and  some  seeded  in,  are  reported  in  Figure  1(b). 
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(a)  Protocol  Stack. 

Fig.  1.  The  TCI  protocol  stack  and  the  number  of  iterations  of  global  state  exploration 
to  discover  the  error. 


Results  on  BDD  sizes  and  discussion.  In  order  to  isolate  the  unpredictable 
effect  of  dynamic  variable  ordering  on  the  BDD  sizes,  we  conducted,  for  each 
error,  two  sets  of  experiments.  In  the  first  set  of  experiments,  we  turned  off  dy¬ 
namic  variable  ordering,  but  supplied  good  initial  orders.  In  the  second,  dynamic 
variable  ordering  was  turned  on,  and  a  random  initial  order  was  given.  Since  the 
maximum  BDD  size  is  often  the  limiting  factor  in  formal  verification,  we  give 
results  based  on  the  maximum  number  of  BDD  nodes  encountered  during  ver¬ 
ification  process,  taking  into  account  the  BDDs  composing  the  controllability 
predicates,  the  reachability  predicate,  and  the  transition  relation  of  the  system 
under  consideration.  We  only  compare  our  results  for  the  verification  using  lazy 
controllability  and  global  state  exploration,  since  these  are  the  most  significant 
comparisons.  Due  to  space  constraint,  we  give  results  for  model  3  of  the  demar¬ 
cation  protocol  as  well  as  the  TCI  protocol. 

Without  dynamic  variable  ordering.  For  each  error,  we  recorded  the  maximum 
number  of  BDD  nodes  allocated  by  the  BDD  manager  encountered  during  verifi¬ 
cation  process.  The  results  given  in  Table  2(a)  and  2(b)  are  the  averages  of  four 
experiment  runs,  each  with  a  different  initial  variable  order.  They  show  that 
often  the  computation  of  the  controllability  predicates  helps  reduce  the  total 
amount  of  required  memory  by  about  10-20%.  The  reason  for  this  savings  can 
be  attributed  to  the  fact  that  fewer  iterations  in  global  state  exploration  avoids 
the  possible  BDD  blow-up  in  subsequent  post-image  computation. 

With  dynamic  variable  ordering.  The  analysis  on  BDD  performance  is  more 
difficult  if  dynamic  variable  ordering  is  used.  We  present  the  results  in  Tables  2(c) 
and  2(d)  which  show  the  averages  of  nine  experiment  runs  on  the  same  models 
with  dynamic  variable  ordering  on.  Dynamic  variable  ordering  tries  to  minimize 
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(a)  Demarcation  Protocol  (Off). 
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(b)  TCI  (Off). 
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(c)  Demarcation  Protocol  (On). 

(d)  TCI  (On). 

Table  2.  Average  maximum  number  of  BDD  nodes  required  for  error  detection  during 
the  controllability  (Control)  and  reachability  computation  (Total)  phases.  Dynamic 
variable  ordering  was  turned  off  in  (a)  and  (b),  and  on  in  (c)  and  (d).  The  results  are 
given  for  lazy  controllability  and  global  state  exploration.  All  data  are  in  thousands  of 
BDD  nodes,  and  the  standard  deviations  are  given  in  parenthesis. 


the  total  size  of  all  the  BDDs,  taking  into  account  the  BDDs  representing  the 
controllability  and  the  reachability  predicates,  as  well  as  the  BDDs  encoding 
the  transition  relation  of  the  system.  Hence,  if  the  BDDs  for  the  controllability 
predicates  are  a  sizeable  fraction  of  the  other  BDDs,  their  presence  slows  down 
the  reordering  process,  and  hampers  the  ability  of  the  reordering  process  to 
reduce  the  size  of  the  BDD  of  the  reachability  predicate.  Thus,  while  our  methods 
consistently  reduce  the  number  of  iterations  required  in  global  state  exploration 
to  discover  the  error,  occasionally  we  do  not  achieve  savings  in  terms  of  memory 
requirements. 

When  the  controllability  predicates  are  small  compared  to  the  reachability 
predicate,  they  do  not  interfere  with  the  variable  ordering  algorithm.  This  ob¬ 
servation  suggests  the  following  heuristics:  one  can  alternate  the  iterations  in 
the  computation  of  the  controllability  and  reachability  predicates  in  the  fol¬ 
lowing  manner.  At  each  iteration,  the  iteration  in  the  controllability  predicate 
is  computed  only  when  its  size  is  smaller  than  a  threshold  fraction  (say,  50%) 
of  the  reachability  predicate.  Otherwise,  reachability  iterations  are  carried  out. 
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Another  possible  heuristics  to  reduce  the  size  of  the  BDD  representation  of  the 
the  controllability  predicates  is  to  allow  approximations:  our  algorithms  remain 
sound  and  complete  as  long  as  we  use  over-approximations  of  the  controllability 
predicates. 

6  Bounded  Controllability  and  Iterative  Strengthening 

Bounded  controllability.  In  lazy  controllability,  we  know  that  there  is  a  move 
of  the  environment  that  is  always  enabled  (the  move  that  leaves  all  external 
variables  unchanged);  therefore,  that  move  must  be  able  to  control  the  mod¬ 
ule.  In  constrained  controllability,  we  are  given  the  set  of  possible  environment 
moves,  and  we  require  that  one  of  those  moves  is  able  to  control  the  module. 
We  can  combine  these  two  notions  in  the  definition  of  bounded  controllability.  In 
bounded  controllability,  unlike  in  usual  games,  the  environment  may  have  some 
degree  of  insuppressible  internal  nondeterminism.  For  each  state,  we  are  given  a 
(nonempty)  set  A  of  possible  environment  moves,  as  in  usual  games.  In  addition, 
we  are  also  given  a  (possibly  empty)  set  B  C  A  of  moves  that  the  environment 
can  take  at  its  discretion,  even  if  they  are  not  the  best  moves  to  control  the 
module.  We  say  that  a  state  is  boundedly  controllable  if  (a)  there  is  a  move  in  A 
that  can  control  the  state,  and  (b)  all  the  moves  in  B  can  control  the  state.  The 
name  bounded  controllability  is  derived  from  the  fact  that  the  sets  B  and  A  are 
the  lower  and  upper  bounds  of  the  internal  nondeterminism  of  the  controller. 

Given  a  module  P,  we  can  specify  the  lower  and  upper  bounds  for  the  en¬ 
vironment  nondeterminism  using  two  predicates  Hl,Hu  £  V(Vp  U  £'P).  We  can 
then  define  the  bounded  uncontrollable  predecessor  operator  BUPre[f?z,  Hu]  : 
V(VP)  ^  V(VP)  by 

BUPr e[Hl,Hu](X)  =  [\/£'P.{Hu  -+  3C'P.{TpAX'))\ Xf[3£'P.{Hl  /\3C'p.(TP/\X'))]  . 

Note  that  the  quantifiers  are  the  duals  of  the  ones  in  our  informal  definition,  since 
this  operator  computes  the  uncontrollable  states,  rather  than  the  controllable 
ones.  Note  also  that  in  general  we  cannot  eliminate  the  first  disjunct,  unless 
we  know  that  3£P  .  Hl  holds  at  all  s  £  States  (P),  as  was  the  case  for  lazy 
controllability.  By  substituting  this  predecessor  operator  to  UPre  in  Algorithm  1, 
given  a  predicate  over  Vp,  we  can  compute  the  predicate  BCtr[H' ,  HU](P,  □</?) 
defining  the  states  of  P  that  are  boundedly  controllable  w.r.t.  □</?.  Given  a  system 
R  =  Pi  ||  •  •  •  ||  Pn  and  a  predicate  ip  over  Vp,  we  can  use  bounded  controllability 
to  compute  a  set  of  doomed  states  as  follows.  For  each  1  <  i  <  n,  we  let  as  usual 
absi(ip)  =  3 (Vr  \  Vp;)  .  p>,  and  we  compute  the  lower  and  upper  bounds  by 

H\  =  Aje{l,...,n}\{«}^.«  ’  ■  Tpi  >  =  AjG{l,...,n}\{i}3^>*  '  ’  Tpj  > 

where  for  1  <  j  <  n,  the  set  Ujj  =  VPj  \  VPi  consists  of  the  variable  of 
Pj  not  present  in  Pj.  We  can  then  check  whether  R  |=  Utp  by  executing 
InvCheck(R,  A  A^Li  BCtr[H\,  Hf]{Pi,  Uabsi{ip))).  If  this  check  fails,  we  can 
construct  counterexamples  by  proceeding  as  in  Section  3. 
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Iterative  strengthening.  We  can  further  strengthen  the  controllability  pred¬ 
icates  by  the  process  of  iterative  strengthening.  This  process  is  based  on  the 
following  observation.  In  the  system  R  =  Pi  ||  •  •  •  ||  Pn,  in  order  to  control  Pi, 
the  environment  of  Pi  must  not  only  take  transitions  compatible  with  the  tran¬ 
sition  relation  of  the  modules  Pj,  for  j  €  {1, _ ,n}  \  {i},  but  these  modules 

must  also  stay  in  their  own  sets  of  controllable  states.  This  suggests  that  when 
we  compute  the  controllable  states  of  Pi,  we  take  into  account  the  controllability 
predicates  already  computed  for  the  other  modules.  For  1  <  i  <  n,  if  Si  is  the 
controllability  predicate  of  module  Pi,  we  can  compute  the  upper  bound  to  the 
environment  nondeterminism  by 

H?(6)  =  Aje{i ,...,„}\{i}3Z4i  •  3Uj,i  •  (TPj  A  Si  A  S't)  , 

where  S  =  S\,...,5n.  For  all  1  <  i  <  n,  we  can  compute  a  sequence  of  in¬ 
creasingly  strong  controllability  predicates  by  letting  S®  =  T  and,  for  k  >  0,  by 
S^+1  =  BCtr[H\,  Hf{Sk)\{Pi,  □<£>).  For  all  1  <  i  <  n  and  all  k  >  0,  predicate 
<5^+1  is  at  least  as  strong  as  df.  We  can  terminate  the  computation  at  any  k  >  0 
(reaching  a  fixpoint  is  not  needed),  and  we  can  verify  R  |=  □  ip  by  executing 
InvCheck(R,  A  A"=i  £?)•  As  &  increases,  so  does  the  cost  of  computing  these 
predicates.  However,  this  increase  may  be  offset  by  the  faster  detection  of  errors 
in  the  global  state-exploration  phase. 

Discussion.  The  early  error  detection  techniques  presented  in  the  previous  sec¬ 
tions  for  invariants  can  be  straightforwardly  extended  to  general  linear  temporal- 
logic  properties.  Given  a  system  R  =  Pi  \\  ■  ■  ■  \\Pn  and  a  general  LTL  formula  ip 
over  Vr,  we  first  compute  for  each  1  <  i  <  n  the  predicate  Si,  defining  the  con¬ 
trollable  states  of  P{  with  respect  to  ip.  This  computation  requires  the  solution 
of  w-regular  games  [EJ91,Tho95];  in  the  solution,  we  can  use  the  various  notions 
of  controllability  developed  in  this  paper,  such  as  lazy,  constrained,  or  bounded 
controllability.  Then,  we  check  whether  R  (=  'ip  A  □(A"=i  &):  as  before,  if  a  state 
that  falsifies  Si  for  some  1  <  i  <  n  is  entered,  we  can  immediately  conclude  that 
R  ip.  For  certain  classes  of  properties,  such  as  reachability  properties,  it  is  con¬ 
venient  to  perform  this  check  in  two  steps,  first  checking  that  R  (=  □(A"=i  ^») 
(enabling  early  error  detection)  and  then  checking  that  R\=  tp. 

Acknowledgements.  We  thank  Andreas  Kuehlmann  for  pointing  out  the  con¬ 
nection  of  this  work  with  target  enlargement. 
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